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5 FIELD OF THE INVENTION 

The present invention relates to a system and method for reactive animation, 
and in particular to such a system and method in which events are generated and 
evaluated by an event driven en^e for the execution of reactive gr^hic animation by 
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BACKGROUND OF THE INVENTION 

Animation of complex behaviors is very difficult to create, for example for 
animation of a complex situation, and/or for adding complex behavior to animation. 
Currently, animation is predetermined, in that the exact behavior of the animation 
15 must be known and created in advance. Animation cannot be created by allowing 
different objects to freely interact, and then creating the animation from such 
interactions. 

One example of a situation which features complex behaviors is for 
interpreting sets of data in science. The task of collecting data and displaying it in a 
20 readily comprehensible format is complex, particularly for dynamic data. For 
example, scientific papers, often the sources of such dynamic data, may provide a 
complex dataset for a given phenomena in text, table, and figures that are difficult to 



an animation engine. 
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translate into other media. The language used in scientific papers is usuaUy 
comprehensible only to the specific field of research. 

For example a specific scientific topic may be leseaiched in different fields, 
for example cell migration and cell differentiation, which may inchide data from 
5 histology, electron microscopy, biochemistry, and molecular biology. Each field 
forms a specific viewpoint and dateset Therefore, constructing representations of 
models which are usefiU for comprehension by scientists in oAer fields and/or which 
are usefiil for phenomena which are not restricted to a particular field, can be qiiite 
difficult 

10 One particularly difficult problem is dynamic modeling of a system, which 

requires that changes to a system be both modeled and represented. The 
representation of a dynamic model requires that changes over time be accurately 
depicted, in a manner which is comprehensible to other scientists. 

Tbeoretically, animation of a model could solve this problem, by allovwng 
15 dynamic, changing representations of the model to be easily displayed. Unfortunately, 
animation is currently produced frame by frame, thereby giving the illusion that tiie 
objects in die animation perform some interactions among themselves. The objects in 
tiiis "standard" animation do not sense each other and Ihek behavior is not tiie 
outcome of some defined description of stimuli and response. Therefore, this ^pe of 
20 representation is not useftd for a model, as it is predetermined and does not reflect 
dynamic changes in the model itself. 

Furthermore, animation in general suffers from this limitation, as it limits the 
possible interactions between objects to those which have been previously determined. 
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* * 

^ For computer games, for example, such a limitation restricts the ability of a player to 
interact with the game, and also limits the possible of a realistic representation of a 
virtual world in the game. This limitation also increases the cost of creating 
animation, as all visual depictions must be predetermined and manually programmed 
5 in the scripting language of the animation tool, rather than allowing such 

representations to arise naturally as a result of interactions between the objects. 

SUMMARY OF THE INVENTION 

The present invention is of a system and method for generating reactive 

10 animation involved in providing a generic link between tools for the specification and 
execution of reactive systems and tools for graphic animation. The present invention 
can optionally be used in a wide range of ^plications, including computer games, 
navigation and traffic systems, physiology models, and interactive scientific 
animation. Reactive animation helps make the prograiruning of such applications 

15 more reliable, expeditious and natural. The present invention seeks to operatively link 
the representation of an event driven system in conjunction with an event driven 
engine to an animation engine in order to generate reactive animation. Such a 
combination has not been taught or suggested by the background art, since animation 
has previously been created according to predetermined methods. 

20 In an optional preferred embodiment of the present invention, the event driven 

system may be a statefii] system and the event driven engine may be a state engine. In 
an alternative optional preferred embodiment of the present invention, the event 
driven system may be a sequence of action (scenario based) system and the event 
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option^ d«»attve p«*™d «nbodtaeot of fte ptcscn. invention, the event driven 
system may be a temporal (time based) system and the event driven engine may be a 
temporal (toe based) enpne. Tbc above-mentioned embodiments are by way of 
example only and are not meant to be limiting. 

A stateftal system, a system in «hibh the past history of the system's objects 
impact ftatnre bd,aviot of those objeets and the system as a whole i^ weU knovm in the 
art Ammatioo as a means of graphical representation of predetermined systems is 

well known in the art. 

to ttie optional preferred embodiment of a stateM system, the behavior of 
objects within a given system is modeled within a reactive animation enviromnent. 
Objects in a reactive animation enviromnent are preferably mapped to physical 
objects, CMT optionally conc€i>tual objects. 

states may prefe^bly be logieaUy mm^ to animation primiUves. simple 

15 actions carded out by ol^ects which nm ptrf«*ly not *™» ^ 

actions. «a which a«refore preferably represent the simplest or most basic acttons or 
p,«ti«» of the fflrimation. The animaBon is then created by translating each such state 
into a visual depiction of the action. It should be noted Aat an action may optionaUy 

be descriptive of the object. 
20 For example, within the context of computer games, the animation would 

require modeling of movement on the behalf of characters and places, as well as 
visually descriptive aspects of the characters and places, and possibly even the 
background. 
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^ States describe the behavior of each piece of animation separately. Therefore, 

the animation engine does not need to be cognizant of what is happening in the scene, 
but only how to arrange all of the requested pieces of animation. The animation 
engine makes the animation look realistic. Computer games had previously focused 
5 on the appearance of realistic-looking behavior within the game, but were not 
concerned witfi actually modeling realistic behavior. 

Preferably, in addition to states representing physical changes such as color 
changes, states may also preferably be mapped to mteractions between objects. For 
example, if an object representing a fist impacts an object representing a piece of 

10 paper, a transitional interactive state of "fist punching paper" which models the 
animation of a fist punching a piece of paper may be generated which has a very 
limited life span (e.g. 0.3 seconds) and then dies. 

After the transitional interactive state of "fist punching paper^ has died, the 
behavior of the fist and the piece of paper may continue to be modeled by states 

15 representing the fist and states representing the piece of paper respectively. States 
modeling the behavior of the fist are preferably mapped to animation primitives 
depicting the fist, which are output onto a display device. States modeling the 
behavior of the piece of paper are preferably mapped to animation primitives 
depicting the piece of paper which are output onto a display device. 

20 Tools for the execution of reactive systems via state engines are well known in 

the art. Rhapsody™ fi-om I-Logix, Inc. is an example of a eommercially available 
state engine. Graphical animation tools are also well known in the art Flash™ from 
Macromedia is an example of a commercially available animation engine. By way of 
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example only, and without any intention of being limiting, tbe present invention is 
demonstrated with Rhapsody™ as the state engine and Flash™ as the animation 
en^ne. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is herein described, by way of example only, with reference to the 

accompanying drawings, wherein: 

FIG. 1 is a schematic block diagram of a preferred exemplary reactive 

animation system according to die present invention; 
10 FIG. 2 is a graphical representation of a UML model depicting a preferred 

reactive animation system according to the preferred embodiment of the present 

invention; 

FIG. 3 is a schematic depiction of layering within a preferred reactive 
animation system; 

15 FIG. 4 is a schematic block diagram of an exemplary preferred reactive 

animation method; 

FIG. 5 is a schematic block diagram of an exemplary preferred reactive 
animation method demonstratmg the building of a plurality of actor and actor- 
component movie clips in the animation engine; 
20 FIG. 6 is a schematic block diagram of an exemplary preferred reactive 

animation method demonstrating the building of a sunulation/movie 
/execution/running model in the event driven engine; 
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FIG. 7 shows part of the actual simulation of a thymus model according to the 
present invention; and 

FIG. 8 shows an exemplaiy menu according to the present invention. 

5 DESCRIPTION OF THE PREFERRED RMBODIMENTS 

The present invention is of a method for providing animation, by determining 
the behavior of an object according to a plurality of events with an event driven 
engine, and the visual depiction of that behavior as a plurality of portions of animation 
by an animation en^e. Preferably, an interface translates the information from i^e 

10 event driven engine concerning the event which acted upon the object, into one or 
more conmiands for the animation engine. The animation engine then renders one or 
more portions of animation for display. 

In one preferred embodiment of the present invention a method for producing 
animation of an object includes: modeling a behavior of the object as a plurality of 

15 events, deterinining at least one transition between the plurality of events, creating a 
visual depiction which at least represents the plurality of events, detecting an event 
which acts upon the object, and creating the animation according to the event with the 
corresponding visual depiction. 

In an optional preferred embodiment of the present invention, the event driven 

20 system may be a statefiil system and the event driven engine may be a state engine. In 
an alternative optional preferred embodiment of the present invention, the event 
driven system may be a sequence of action (scenario based) system and the event 
driven engine may be a sequence of action (scenario based) engine. In another 
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' optional alternative preferred embodiment of the present invention, the evrat driven 
system may be a temporal (time based) syst^ and the event driven en^e may be a 
temporal (time based) engine. The above-mentioned embodiments are by way of 
example only and are not meant to be limiting. In the optional preferred embodiment 

5 of the present invention, the event driven system may be a statefiil system and tiie 
event driven engine may be a state engine* Within the optional preferred embodiment 
of the present invention in which the event driven en^e is a state engine, at least one 
transition may preferably be determined according to at least one rule. 

Furthermore within the method described above, die visual depiction may 

1 0 preferably be created to further a visual depiction of at least one transition. 

Within a preferred embodiment of the present invention, events cause the event 
driven engine to potentially react. At the very least, the event driven engine will 
evaluate whether the event driven engine must take action. Though, any given event 
may not cause a change of status for an object, the event driven engine must at least 

15 evaluate whether or not the status of the event should be changed in response to that 
event. For example in a stateful system, the state of an object may not change in 
response to each stimulus. 

Events, both external and internal events, represent stimuli to the engine. 
Internal events occur when there has been a change within die event driven engine. 

20 Some other factor affected a change within the event driven engine for whatever 
reason and that change generates an event. 
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External events occur when external stimuli act upon the event driven engine 
within ftte reactive animation system. For example, user input entered throu^ a user 
interface may act as a means of generating external events- 

Wilhin an exemplary, and qptional embodiment of an event driven eng^e, 
5 there may preferably be different sequences of events (scenarios). An event occurs 
and the event enguie evaluates whether a change should occur. If the event does not 
match to any event which has an effect, the event driven engine may preferably not 
effect a change. For example, within a scoa^o based system action follows action 
and possible scenarios could be, the ball comes to a soccer player, Ihe soccer player 
10 hits the ball, and the ball moves somewhwe else. 

In a stateful system, every state needs to be mapped. For example to represent 
the behavior of a soccer player in a stateful system all of the states that Soccer player 
can be m must be represented, for example: the soccer player hitting the ball, the 
soccer player kicking the ball, the soccer player running with the ball, and the soccer 
15 player trying to get the ball. 

* In anotiier alternative preferred embodiment of the present invention, the event 
driven engine may be a temporal logic engjne and the reactive animation may be 
modeled by a temporal system, a system in which the status of objects within a system 
are sampled at discrete moments in time. 
20 In a computer game, there may be more than one sequence, but the sequences 

are set. However in alternate preferred embodiments of the present invention, there 
may preferably be multiple sequences or multiple scenarios, multiple states, or 
multiple objects being sampled with respect to time. 
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In an exemplary, but preferred embodiment of the present invention a system 
for producing animation of an object is modeled, in which a behavior of the object is 
modeled as a plurality of states, including: a state engine for modeling the plurality of 
states and at least one transition between the plurality of states, an animation engine 
5 for creating a visual representation at least of each of the plurality of states, and an 
interface for receiving a state of the object &om the state engine, and for sending a 
command to the animation engme for creating the visual depiction according to the 
state. Within a given stateful system, each state may be entered from multiple other 
states. For example an "end of game" state may preferably be entered at the end of all 

1 0 possible paths within a given reactive animation system. 

In order to assist in diagramming the states and determining the state of an 
object, Statecharts may optionally be used. Statecharts is a visual language, invented 
by D. Harel in 1984 [17, 18] to assist ha the development of the avionics system of a 
new aircraft. Statecharts has since become the subject of research for many groups 

15 [19, 20, 2 1 , 22] as the main formalism used to specify the behavior of complex 

reactive systems in a variety of industries, and has been adopted as the central medium 
for describing behavior in the Unified Modeling Language (UML), a world standard 
for object-oriented specification and design of systems [23]. 

Behavior in Statecharts is described using states and events that cause 

20 transitions between states. States may contain sub-states thus enabling description at 
multiple levels, and zooming in and zooming out between leveb. States may also be 
divided into orthogonal states, thus modeling concurrency, allowing the system to 
reside simultaneously in several different states. Statecharts are rigorous and 
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^ mafhematically well defined and are therefore amenable to execution by computers. 
Sev^^ tools have been built to support Statecharts- based modeling and execution, 
and to enable automatic translation from Statecharts to machine code. 

Within tiie system described above, the state engine preferably comprises 
5 Rhapsody™ [24] and the animation engine preferably comprises Flash™ as non- 
limiting examples thereof 

Preferably within the system described above, there is an extemal input 
module for sending a command to the interface for interacting with the object. 

Additionally within the system described above, the extemal input module 
1 0 comprises a user interface. 

The principles and operation of the present invention may be better understood 
with reference to the drawings and the accompanying description. 

Referring now to the drawings, FIG. 1 is a schematic block diagram of a 
preferred exemplary reactive animation system according to the present invention. 
15 Reactive animation system 100 comprises a state engine 1 10, an interface 120, 

and an animation engine 130. The state engine 110 fiirther comprises a state 
processor 140 and at least one statechart 150 (preferably written in the language, 
Statecharts). Preferably each object has an associated statechart 150, such that there is 
a plurality of statecharts 150. 
20 The interface 120 further comprises a conunand parser 160 and a command 

translator 170. The animation engine 130 further comprises a plurality of animation 
pieces 180, a rendering engine 190, and an input translator 200. A user mterface 210, 
whereby a user (not shown) may preferably interact with the reactive animation 
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system 100 may be operatively linked to the animation engine 130 or comprised 
witiiin the soiiniation ea0n& 130. 

In alteinate embodiments the user inter&ce 210 naay be comprised within the 
interface 120 or operatively associated with the interfaee 210. 
5 In a preferred embodiment the user interface 210 is preferably implemented as 

a gr£q)hical user inter&ce (GUI). When the user clicks on part of the screen, (for 
exanq>le when working in conjunction with an operating system such as the 
Windows™ (Microsoft Corp USA) operating system, which can mterpret mouse 
clicks), the animation en^me 130 may preferably recognize that an event occurred 
10 where a mouse click occurred at these coordinates. 

Optionally and preferably the animation engine 130 is implemented as a Flash 
animation engine, or a similar type of engme with similar capabilities tiiat are known 
in tiie art. The Flash engine already has flie built in capability to interpret user inputs 
under certain basic circumstances. Flash understands what these inputs mean with 
15 respect to tike animation. Flash may flien send a command back to the interface 120, 
informing die interface 120 that a click occurred' and sending a query for further 
instructions. 

In an alternative embodiment, the input translator 200 is operatively 
associated or comprised withm the mterface 120, in which case, the interface 120 has 
20 to determine two types of information. First, flie user interface 210 needs to be aware 
of events caused or driven by the user. For example for a mouse click, the screen 
coordinates form part of the data for the event. The user interface 210 also has to be 
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• aware of the actions of the animation itself, in order to determine the meaning of the 
coordinates of the mouse click. 

H^ce it is preferable for the user interface 210 to be comprised within or 
operatively associated with animation engines that abeady have at least a limited 
5 ability to interpret events fiom the user interface 210. 

Animation engine 130 preferably has scripting capabilities, such that 
commands may be passed to animation engine 130 in some type of scripting language. 
These capabilities are well known in the art, and are provided by many different 
commercially available animation software programs. Animation engine 130 should 

1 0 preferably be vector based, which is also well known in the art. Vector based 

animation engines describe objects being displayed on a computer monitor or other 
type of display as a vector, rather than as a bitmap (which is an actual image). Vector 
based animation can be described in terms of changes to the display, rather than 
describing each part of the animation as it changes on the display. Such animation is 

1 5 much simpler to program. 

Within the state engine 1 10, the state processor 140 handles states and state 
transitions, and is able to cause a transition upon receiving input data. By "causing a 
transition" it is meant that the state of an object is changed from a first state to a 
second state. In order to assist in the determination of such transitions, state processor 

20 1 40 preferably comprises software that performs the primary and highly repetitive 
function of mapping input data (not shown) to various states and commands and 
inputting the relationships between input data and states into the statechait 150. 
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Each of a plurality of objects 220 is preferably mapped to one of a plurality of 
statecharts 150. Each of the plurality of objects 220 represents a plurality of states of 
an object within an animated environment. For example a subset of the plurality of 
states for the hand object in a video game reactive animation dataset might comprise a 

S state for a hand at rest, a state for a hand waving, a state for a hand forming a fist, a 
state for a fist at rest, a state for a fist swinging forward, a state for a fist recoiling 
backwards, a state for a hand holding a small object, a state for a hand holding a 
medium object, and a state for a hand holding a large object. 

The state engine 110 may preferably not be cognizant of the fact that the state 

10 engine 1 10 is modeling animation information. The state engine 1 10 is used by the 
user (not shown) to logically represent the relationships between the plurality of 
objects 220, and the set of commands (not shown) input by the user that governs the 
means by which the plurality of objects 220 interact within the animation 
environment. 

IS The interface 120 is the part of the system that enables the state engine 110 

and the animation engine 130 to communicate. Within the int^face 120, the 
command parser 160 interprets the commands generated by the state engine 1 10 via 
the construction of commands by the state processor 140 by reading the statechart(s) 
150. 

20 The command translator 170 interprets the parsed output of command parser 

160 from a form that is intelligible to the state engine 1 10 into a form that is 
intelligible to the animation engine 130, preferably producing a plurality of animation 
pieces 180 and a set of animation instructions (not shown) which the rendering engine 
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190 utilizes to produce a reactive animation model on a display device (not shown). 
By way of example and without being limiting, the display device may comprise any 
display device as is well known in the art including a television screen and a computer 
screm. 

5 In some preferred embodiments of Ihe present invention, the user (not shown) 

may not interact with the reactive animation model. In other preferred embodiments 
of the present invention, the user may interact with the reactive animation model via 
an optional user interface 210. 

In one preferred embodiment of the present invention, user inputs entered via 

10 the user interface 210 are translated by the input translator 200 and may preferably 
directly change the way objects are displayed on the display device by changing the 
appropriate parts of the plurality of the animations pieces 180. The input translator 
200 preferably communicates with animation engine 130, for reasons described in 
greater detail above, which then sends commands and/or information to command 

1 5 parser 160 about the change(s) that have occurred. 

The command translator 170 then interprets these changes into data that can be 
understood by the state engine 1 10* The state processor 140 then makes the 
appropriate change(s) in the current situation of the appropriate statechart(s) ISO. 
In another alternative preferred embodiment, the user interface 210 may 

20 communicate directly with the interface 120 via the input translator 200 (positioned in 
the interface 120 instead of the animation engine 130). The input translator 200 
would then route translated user input data to the command translator 170 in the 
interface 120. The command translator 120 would then translate and route the data 
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representing the user initiated changes to both the state engine 1 10 and the animation 
engine 130 in a form Avhich both the state engine 110 and the animation engine 130 
could respectively utilize. Both the state engine 1 10 and the animation engine 130 
would alter the data as requested by the user via conventional internal means as is well 
5 known in the art. Optionally, user interface 2 10 may be operatively associated with 
state engine i 10. 

FIG. 2 is a graphical representation of a portion of a UML model depicting a 
prefeired reactive animation system according to the preferred embodiment of the 
present invention. 

10 By way of example only, without any intention of being limiting, FIG. 2 

illustrates a portion of a Unified Modeling Language (UML) model 300 of a reactive 
simulation of a car interacting with road traffic. The Unified Modeling Language 
(UML) model 300 comprises at least one statechart 310 and an object model diagram 
320. 

1 5 The statechart 310 represents a stage tag 330 which demonstrates that the car 

interacting with road traffic is in state_7. The state mode 340 demonstrates that the 
car is driving. Within the state mode 340 there are a plurality of state descriptors 350 
which further describe the state mode 340. 

In the example of a car interacting with road traffic in state tag 330 state_7, the 

20 plurality of state descriptors 3S0 comprise an acceleration module 360 and a direction 
module 370. The acceleration module further comprises a plurality of acceleration 
module alternatives 375 comprising accelerating 380, constantspeed 390, and 
decelerating 400. It should be noted that this diagram shows only a few features of 
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the state system for clarity only, as additional states may be desirable for this system. 
Fuith^more, this diagram does not include other classes that may interact with the 
automobile according to their OAvn states, and as such does not completely represent a 
trafSc model. 



either be accelerating, decelerating, or driving at a constant speed. The arrows which 
point between the alternatives accelerating 380, constantspeed 390, and decelerating 
' 400 demonstrate tiiat from any one of the three alternatives mentioned above, the car 
can change to any of the other two alternatives. 



straight 420, or right 430. 

Whenever the scalar quantity of speed is equal to zero within the state mode 
340 of drivmg the car leaves the driving mode 340 of driving and enters the stopped 
submode 440, Wh^ the car is in the stopped submode 440 and the scalar quantity of 

1 5 speed becomes nonzero the car re-enters the state mode 340 of driving. 

The system of the car (not shown) comprises a driver (not shown) who 
interacts with outside stimuli in the traffic simulation. The object model diagram 320 
demonstrates the driver's interaction with outside stimuli. The object model diagram 
320 comprises a scan module 450 which models the driver's behavior via the looking 

20 module 460. Via the looking module 460 the driver scans the surroundings for factors 
of interest comprising a plurality of factors 465 preferably represented by a 
trafficLight 470, a danger flag 480, a carinFront flag 490 representing a car in front of 
the driver's car, and a policeman 500. Data in the statechart(s) 1 50 (of FIG. 1) may 
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Acceleration module 360 demonstrates that when the car is driving, the car can 
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The direction module 370 demonstrates that the car's direction can be left 410, 
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preferably be changed in the state engine 1 10 (of FIG. 1) by the state processor 140 
(of FIG. 1) based upon fhe fector racount^red by the driver as encountered by the 
looking module 460. 

After one of the plurality of Motors 46S is encountered, and any necessary 
5 stage changes are made to the statechart 150, logical control of the object model 
diagram 340 ccmtinues with the looking module 460. 

Another part, of the model is the infrastructure that enables traffic motion. This 
infrastructure is a representation of streets and sidewalks to facilitate movement and 
positioning of all other objects. This infirastructure may be modeled as a two 
10 dimensional lattice that holds the positions, sizes and visual features of the reactive 
objects (cars, people, etc) and stationary objects (roads, buildings, etc). 

FIG. 3 is a schematic depiction of logical steps to construct a preferred 
reactive animation model. 



15 Animation Layer 610 communicates with the Reactivity layer 620 via a series of 
bidirectional messages. The interface 120 (FIG. 1) generates several intermediate 
layers that enable communication between the Animation layer 610 and flie Reactivity 
layer 620 including, but not limited to, top down data 630 and statistical analyses 640. 
The interface 120 also preferably comprises customization data for different viewers 

20 650 which enables a given reactive animation system to be displayed on a plurality of 
different display devices (not shown). 

These different layers support statistical analyses of the behavior of different 
objects and/or sets of objects in the modeled system. For example, each user could 



Logical flow 600 demonstrates layering in reactive animation. 



The 
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decide to focus on different aspects of the modeled system. Also, the different layers 
enable different parts of the present invention to be operated by different computers 
and/or CPUs, further increasing Ihe speed and efficiency of the operation of the 
present invention. 

FIG. 4 is a schematic block diagram of an exemplary preferred reactive 
animation method 700 for a given system. 

In stage 710 a system is specified. For example^ and without intention to be 
limiting, the system may be specified in the form of a UML model (as illustrated in 
Fig. 2) 

The system further comprises stage 720 in which a plmrality of events which 
comprise the system are defined. In stage 720, the plurality of events is represented 
•as a plurality of states. In stage 730, Ibe plurality of states is decomposed into a 
statechart for each object. 

Throughout the system associated with Fig. 4, by way of example only and 
without any intention of being limiting, the plurality of events is represented as a 
plurality of states. 

In stage 740, a visual representation of the defined event stages is designed. In 
Fig. 4, by way of example only and without any intention of being limiting, the 
representation of events is represented as a representation of states. 

In an alternative preferred embodiment of the present invention stage 740 and 
stage 710 may be interchanged in which the plurality of states would be determined 
firom a visual representation. 
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In stage 750, visual landmarks are identified. Identification of visual 
landmaiks is done on the model of the system. In the model, events (i.e. states), 
messages and attributes that are important to the visual representation are identified. 
This stage also involves determining the visual form that an object has in the graphic 

5 animation. For example, the visual form may optionally be more abstract or symbolic, 
or alternatively, more detailed and more realistic. 

In the example of FIG. 2, such states could be the car's motion data, different 
states of a traffic light (colors), different activities of a policeman and different 
activities of pedestrians. A detailed description of traffic may comprise the opening 

10 and closing of a car's door, the motion of a car's steering wheel, the car's light, the 
lighting of the road, the visibility at different conditions and many additional states 
and attributes. 

In stage 760, visual landmarks are associated with states and / or state 
transitions. 

15 The visual landmarks marked in stage 750 are linked to different parameters in 

the model. For example, the car's motion data could be associated with states that 
represent acceleration data, with attributes that stand for the car's speed, and with the 
state that indicates the path of motion (left turn, rigjbt turn). 

In stage 770, formatted messages are preferably determined at least for each 

20 state transition. The states and optionally other parameters are now preferably 
associated with a formal action to execute for one or more state transitions, for 
example for sending a command to the animation engine. Optionally, the messages 
are bi-directional, in which the animation engine may also have one or more messages 
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associated with changes to the animation, for example through input from the user 
iaterface. 

When oos of the patameteo is experienced in the nrodel. d»ri.« «n-time. 
Aat m. e«ot tas ocouned, a message is preferably sent to the animation engine. 
5 For example, in Aetraffie model ofHO. 2. a mess^ which indicates that the 

car «ill torn left wUt be chosen when the driver decides to t«o left The message wiU 
Identifi, the spedfic ear ont of fte entire fleet together wi* an indication to the 
direction of the hm. (in this case "leff^. In the mode^ the car preferably examines the 
sv«omding field of vision, to sample avaUd-le data on «sible irrformation to *is 
10 field. Ihe driver wonid then decide to tarn leS. change lanes or take no action. It 
should be noted that an of Aese «mcq« may t^onally md preferably be 
represented by abstract states, wid. the ultimate action of the car repres«rting a state 

transition to a new state. 

In stage 780. animation components are designed for visually depicting the 

15 states of the system. 

The animation engine works with components that are later activated, moved, 
added and changed according to instructions from the state engine. -Hie design of 
these components can be made using the animation engine or may be imported from 
any graphical design tool. Ihe design of these components is best made while keeping 
20 in mind their reactivity and possible reactions. When an animated component of a car 
is designed using the animation engine, an image of a car is made. The animated 
component of a car may be attached to an animation of a car's move to the left and a 
car's move to the right The more detailed description would include the opening and 
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closing of doors, details of the steering wheel's movements, the dashboard, and 
different lights and so on. 

In stage 790, animation components are assigned to the visual representation 
of the states using scripting language. Animation engines usually include a scripting 
5 language which provides a limited language which comprises the option to apply 
simple behavior to various components. The scripting language does not make it 
possible to apply complicated behavior to components, but only to control the motion 
and appearance of animated objects through non-sophisticated commands. In the 
example of FIG. 2, a car may have, at a specific moment in time, a fixed speed. To 

10 create the illusion of a moving car with the animation engine, a script can be encoded 
which creates the illusion of the rotation of the wheels. The script may receive data 
about the car's velocity and manipulate the images that display the car*s wheels. This 
way, the effect of a rotating wheel is demonstrated but the location and angle of the 
car's wheels do not need to be continuously updated. 

15 In stage 800, functionality is. defined which enables the interface to send 

messages and receive messages from the state engine and the animation en^ne. 

The functionality defmed by stage 800 is preferably a module that may be 
written in the animation engine. This module makes use of the connective capability 
of the animation engine. In the example of FIG. 2, Flash's XML socket, enables 

20 XML files to be sent and received through a TCP/IP connection. Other tools 
preferably comprise other communications options. Every animation tool that 
comprises connectivity to other tools may preferably be configured, using a proper 
addition of modules, to perform stage 800. 
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In stag^ 810, functionality is defined which enables the interface to parse 
messages. Reodved messages must be parsed to make se^se of tiie information they 
convey. For exan^lc an XML message of the foim <CAR INDEX = nO» 
SPEED=-90" TURN="L" > should be parsed and interpreted as "The 10th car, going 
5 90, is turning left". Hie parsing is a convention we, as derigners, agree upon as a 
common language between tools, m message received during animation-time at the 
Animation en^e will be dynamically built according to the dynamics of the nmning 
model. 

In stage 820. parsed messages sent &om the state engine arfe translated and 
10 ^plied . to the animation to control animation components. Parsed messages 
ultimately control the components of &e aiiimation. Whenever a relevant component 
or animation clip should be added, changed or removed, a proper function is able to 
bring this behavior into animated realism. Functions are specifically designed to 
perform different animations. These functions are invoked by parsed messages sent 

15 by the reactive modeling tool. 

In stage 830. a channel of communication is applied to the state enghie, 
animation en^ne and interface. Tliere are various alternatives for buUding a channel 
of communication between tire state engine and the animation engine. According to 
the particular.choice of communication interfece (TCP/IP. windows API. etc ...). the 
20 channel of communication can be optimized. In the example of FIG. 2, TCP/IP is 
used. With TCP/IP as a choice of communication channel, the hardware may be 
easily distributed, since most hardware is built to interact through TCP/IP- 
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In stage S40, a synchronization of data between the state engine and the 
animation ragine is perfonned. The beginnmg of a running simulation may 
preferably be initialized according to a predefined state setup. For example, the 
simulation may have initially been populated with a populated fleet of traffic, located 

5 in a set of roads. Alternatively the state set up may be populated as the simulation 
runs. Different cars may be assigned cUfferent speeds according to some predejBned 
distribution, or depending upon road conditions. The data structure, optionally as well 
as other fectors define how the state mgine and animation engine coordinate with 
each other for synchronization. Examples of these other factors include but are not 

10 limited to, ^tor handling and simulation parameters. 

In stage 850, user input is interactively taken into account in determining state 
data and animation data. 



15 of T-cell behavior in the thymus, can be modeled with a state engine and animation 
engine in tandem through an interface which enables the state engine and animation 
engine to communicate with each other, as described with regard to the illustrative, 
non-limiting examples below. 



The behavior of many reactive systems, including a complex biological model 
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EXAMPLE 1 



IMPLEMENTATION WITH A MODEL 
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Referaice is now made to Figure 5, which is a schematic block diagram of an 
exemplary preferred reactive animation method dononstrating the building of a 
plurality of actor and actor-component movie clips in the animation engine. 

Reference is also now made to Figure 6, which is a sdiematic block diagram 
5 of an exemplary preferred reactive animation method demonstrating the conshiiction 
of a simulation/movie /execution/running model in the event driven en^e. 

The block diagrams of Fig. 5 and Fig' 6 demonstrate the use of the present 
invention to construct a film portion with an example of a man walking his dog. The 
interaction between the event driven engine (a state ragme in the present example), 
10 and the animation tool or animation engine, is performed substantially as previously 
described. More specifically, to behaviorally direct animation firom the event driven 
engine, the ns&c should: 

I. Biuld actor and actor-component movie clips in die animation en^e 

(demonstrated in Fig. 5) 
15 n. Build the sunulation/movie/execution/running-model in the event driven 

en^e 

(demonstrated in Fig. 6) 

The method which generates the actor and actor-component movie clips of Fig. 

S is genially designated 900. 
20 In stage 905, the actors are built. The man, the dog, the street, the leash, other 

dogs or other people that may be encountered are graphically designed. 

In stage 910, movie clips (for parts of the movie or for the whole movie) are 
built for any kmd of behavior the actors may encounter and respond to. For example, a 
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mo^de clip for flie dog wagging its tail; for (he leash; for the dog's bending and the 
man's walking may be built 

In stage 9 1 5^ the animated behavior in response to stimuli is dictated. For 
example, the command 'Vag tail'' coming from the event driven engine should have 
an appropriate mechanism in the animation engine that tells the animation to start 
playing the tail wagging movie clip. 

The response to a stimulus should be able to handle features of the stimulus. 
For example, the command *Svag tail" will usually arrive with more data about its 
features: for example, the speed of wagging. 

Any scene in the movie is therefore the result of sequential attachment, 
removal, change of size, change of location, change of angle, change of color, etc. of 
the different movie clips that comprise the whole movie. 

One scene on the movie may be a meeting between two dogs. According to the 
nature of the meeting Opresxunably the odors the dogs exchange), the features of the 
dogs' tails change. If they are two males, for example, the wagging of the tails will be 
of some specific speed, different from the speed induced by a male-female encounter. 

Fig. 6 demonstrates how the simulation/movie/execution/running-model is 
built in the event driven engine. 

The method which demonstrates how die simulation/movie/execution/running- 
model is built is generally designated 1000. 

In stage 1010, the behavior of actors/classes/agents are described and 
designed. This may be done with tools specifically built for the job (e.g. via 
Rhapsody™) or in any other way that is convenient for the designer. 
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The example has objects such as the man, his dog. other people and their dogs, 
flie infrastnictute (the street they walk i^on), trees, cars, etc. 
The behavior of these objects would usually include the methods that make them 
move around, the way they sample their environment (sniff; see, feel, hear). 
5 In stage 1020, the initial conditions are staged. The people and dogs are given 

a designation of whae to be at tiie start. 

In stage 1030, the events which should be sent to the aniihation engme are 

specified. 

For example, movements of the man and dog wiU be reported to the animation 
10 engine, but stages of information processing are not reported (unless needed). 

In stage 1040, the means through which the messages are sent is specified. 
According to the choice of information the designer wants to deliver to the 
animation engine, a format for tiiis information is selected. For example, a convenient 
format for a message about tiie dog's location would be a reference to tiie specific dog 
1 5 object, followed by the new coordinates. 

In stage 1050, the movie is run. 

For example, a sfanple scene firom the movie might be instantiated by the 
interaction of the animation engine and the state engine. Suppose the scenario is 

determined as follows: 
20 "The man is walking his dog using a leash. They meet another dog or detect 

food." 

This scenario provides tiie overall storylme, or description of the fiUn clip. All 
possible eventualities need to be defined in tiie event driven engine. For example, tiie 
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man's ability to walk is detennined according to his movement along the grid, which 
is fhe street. The dog's movement ninst be similarly constructed. 

One example is when the man is using a leash to walk the dog. The physical 
properties of the leash are combined into the specification: it can't be longer than a 
5 predetermined length, it is connected to the man's hand until decided otherwise, it is 
connected on the other side to the dog. Yet the visual description of the movement is 
entirely animation engine based. The arch formed when one end of the leash is 1 
meter higjhi and the other is 0.5 meters high (the hand of the man and the height of the 
dog, accordingly) is entirely the work of the animation engine and should include the 
10 way arches look in the presence of gravity, etc. The way fhe arch changes when one of 
its anchors is moving (the man is walking) is also the work of the animation engine. 
The way the arch stretches to a fiill length when one of its anchors abruptly stops 
moving (the dog stops to sniff) is, again, a movie clip nnplemented in stage (2) of part 
I. 

15 As another example^ consider the 3 dimensional odor space to which the dog 

may respond. If, for example, an odor source (food) is added somewhere in this space 
(somewhere in the street), the influence of this source flows in space according to the 
fluid dynamics of the air in the street. The dynamics are described in the event driven 
engine. If the dog is capable of responding to scents, at one point or another it might 

20 encounter the scent from our odor source (the dog smells the food). If the simulated 
dog is able to navigate by an odor gradient, the dog moves in the direction of the food. 
These calculations are performed in the event driven engine. 
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The animation engine knows nothing of odors, navigation, pulling of the leash 
or the forces applied by the man on his dog, etc. Yet the animation is capable of 
showing the dog walking in the direction of tiie food, because the event driven engine 
continuously sends messages telling the animation engine the location and direction of 
5 the dog. The animation is capable of showing the man's hand stretching when the dog 
abruptly increases its speed, because the movie clip that shows this stretching of the 
hand has been prepared and because a message telling the animation to use the clip 
was sent jfrom the event driven engine when it detected the need to implement this 
movie clip. The animation shows two dogs meeting, stopping and passing each other 
1 0 with then: tails up, because the clips fliat include the visual depiction of these 

interactions have been prepared and can be launched upon demand from the event 
driven engine. 

The event driven engine therefore behaviorally directs the movie, while the 
animation is the collection of movie clips that are the embodiment of the story. 

15 

EXAMPLE 2 
MODELING A BIOLOGICAL SYSTEM 
In this Example, specific analytical data are used to construct an integrated 
dynamic representation. The integration is preferably performed in two integrated 
20 facets: specifying the dataset in a way that makes it amenable to execution on a 

computer; and representing the objects that are explicitiy specified in the first facet, 
which are cells and molecules. The end result is a moving visual simulation of the 
biological process - intuitive, visual and interactive. This animation is precisely 
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executed by the statechart's specification, and not by the user. Further details may 
al30 be found in the attached Appendix. 

To form the first facet, a detailed desoiption of the relevant objects is prepared. 
The task of collecting the data and translating it into a well-defined, executable, 
5 specification is complex in itself. Scientific papers — the sources of the data - provide 
the dataset in text, tables and figures that are difficult to translate into other media. 
The language spoken in biological papers is usually comprehensible only to the 
specific field of research. The goal is to translate this dataset into a generic and usable 
medium, which is referred to as specification. 

10 The specifications derived firom the actual data are used as a set of instructions 

» guiding the simulation. The cellular and molecular agents comprising the system refer, 
as it were, to ttiese instructions to know how to respond to stimuli. The stimuli may be 
interactions with other cells, interactions with other molecules or various internal 
events such as the passage of time. 

15 Stem cells arrive to die thymus fi-om the bone marrow, and the developing T 

cells go through a series of interactions in different locations inside the thymus. The 
processes a single cell goes through take about two weeks [4], during which the cell 
may proliferate into 10^ ofiTspring cells [5]. The thymic environment is anatomically 
divided into lobes and lobules, and the lobules are finther divided into the areas of the 

20 cortex and the medulla. Since the thymic output of mature T cells is the basis of the 
inununological repertoire, the physiological fimction of the thymus is relevant to the 
study of many diseases, specifically AIDS and autoimmune diseases [6, 7, 8, 9]. 
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Different agents constitute the thymus: epithelial cells form a mesh throughout 
the organ and interact ^th developing T cells to activate and regulate many of the 
processes needed for their maturation [10]. Epithelial cells are separated into different 
subtypes by molecular markers or by anatomical locaticm [1 1]. Maoiophages perform 
5 mainly '^ousekeqping;" tasks to clear the thymus of dead cells [12]. Cytokines are the 
molecules responsible for agnaling betweoi the cells. ChemoJcmes are molecules that 
signal cell movement along gradients [13,14]. Short segments of proteins, called 
pqptides, combine with other molecules ^4ajor Histocompalibility Molecules : MHC) 
to induce different T-cell selection events (e-g. [15, 16]). Thymocytes (T cells in the 
10 thymus) express many different surfece molecules that serve for mteractions with 
other cells and molecules. Epithelial cells, macrophages, cytokines, chempkines, 
peptides, tihtymocytes and cell markers arc all further divided into dozens of 
subgroups, which are not detailed here. 

The thymic environment, loaded with these difTerent objects, presents a 
15 challenge to many researchers from different fields who have detailed knowledge of 
some of its parts, but yet wish to comprehend the whole. Consider three scales of 
analysis: molecules, cells and the whole organ. 

The molecules most relevant for researchers of the thymus are chemokines, 
cytokines and receptors on the cell surface. Specialists in cell migration, for example, 
20 study how chemokines cause cell migration. They measure chemokine expression 
levels in different areas of the thymus, on different cells of the thymic stroma and 
record the responses of thymocytes during different stages of their development. 
Biophysicists study the interactions between chemokine receptors and their 
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chemokine ligands at the atomic level. Other researchers study cytokine and their 
influences on events in thymic development. Cytokines are the main vehicle for 
signaling between cells and so are important in almost every process. Other molecules 
allow thymocytes to bind to other cells and to the extra-cellular matrix (ECM). 
5 Other fields of research look at these molecules in a different way. In 

microscopy, molecules are used as markers to distinguish between different cells 
under the microscope. Researchers in signal transduction look at the same molecules 
to see how they influence a cascade of events inside the cell. 

The questions asked at the cellular level are which cells respond to which 
10 stimuli, how many cells of each type are in each thymic area, and how many cell types 
are in various areas. What are the events that will drive a cell toward one fate and not 
anotiier? What stages does a cell go throug^i during development? Where is the cell 
during different stages of development? What are the paths two cells follow when 
they mteract? Which selection events are the most influential? How does mutation 
15 influence cell survival? 

Researchers looking at the thymus as one whole often see the organ as a "black 
box*'. Their questions include: what is the niunber of cells the thymus produces under 
specific conditions? How many cells enter the thymus every day/hour/minute? What 
are the effects of removing the thymus (thymectomy)? Why does the thymus diminish 
20 in size with age? What are the influences of diseases on the thymus and what is the 
influence of the thymus on disease? Are there mathematical formulas that can 
recapture thymic output behavior? 
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Butthe %nios is one whole. Disjointed research parcels the same molecules 
and cells into separate fields, and produces data that must be joined, if we are to ever 
understand T cell maturation in the whole organ. Currently, there is no way to 
integrate this broad spectrum of different types of data into one view that would be as 
5 coherent as the biological environment that produced them. 

The present invention overcomes this disadvantage by using the data generated 
by reductionist Wology and integrating the data into a specification model using 
statecharts as previously described. The model is then executed. The results of Ihe 
execution are used to drive an animated visual image of cells and molecules and their 
10 interactions. This type of representation is ffiendly to human minds, and yet does not 
sacrifice matiiematical precision. Moving cells and molecules are interactive wifli the 
tiioughts of ttie user, and tiie format provides tiie user witii tools to choose specific 
views and mediate particular types of execution. 

Moreover, the representation may optionally be designed to express different 
15 tixeories. Immunology, Uke other complex and incompletely characterized fields, uses 
theories to mtegrate both known and unknown information. Theories are proposed 
scenarios. The model and simulation can accommodate different theories. Whenever 
an interaction takes place, the user (or tiie simulation itself) can choose one tiieory 
from a collection of avaUable tiieories and instantiate tiiat particular tiieory to its 
20 completion in tiie simulation. The instantiated tiieory tiien sends conclusions back to 
flie simulation. The user can choose a particular tiieory eitiier during run-time or 
during specification. The outcomes of various tiieories can be compared and 
contrasted. 
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Specifying the Thymus with Statecharts 
States and transitions as descriptors of cell behavior 
For specification and modeling, as a non-limiting example, the language of 
Statecharts may optionally be used, as previously described. It is not intuitively 
5 obvious that cells and molecules may be naturally described by states and transitions. 
In fact, th«e is no consensus on how one should desoibe cells. However, 
immunolo^sts, whether they know it or not, do use states to describe cells. A cell is 
usually described by the collection of markers it expresses on its sur&ce [25]. For 
example, a T cell is called "double negative'' when neither of the CD4 and CDS 
10 molecules is expressed. A human T cell is referred to as a memory cell when it 
expresses a molecule called CD45RO+ [26] and as a suppressor cell when it co- 
expresses CD25 and CD4 without bemg activated [27, 28, 29, 30, 31]- Immunologists 
call these molecules markers, but the present invention refers to them, during 
specification, as orthogonal states of the cell. One may object to describing cells 
15 according to markers that are not chemically accurate descriptions. However, the 
notation is the basis of most immunological reports and immunological terminology. 

In Statecharts, transitions take the system from one state to another. In cell 
modeling, transitions are the result of biological processes or the result of user 
intervention. A biological process may be the result of an interaction between two 
20 cells, or between a cell and various molecules. 
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Dealing with a large dataset 

Statecharts provide a controllable way to handle the enormous dataset of cell 
behavior by providing the ability to specify separation into orthogonal states and by 
allowing transitions. 



Example 1: Modeling thymocyte movement 
To demonstrate the conversion of data into specification, the way thymocytes 
move in the tfiymiis is followed as a non-limiting example. Thymocytes receive 
signals from different cells in different locations. To be certain that signals are 

10 received at the right time is actually to be certain that the right thymocyte is in the 
right place at the right time. The molecules responsible for directing cells along a 
gradient path are called chemokines. The role of four chemokines are considered: 
TECK, SDF, MDC and SLC. Thymocytes search their environment for chemokines 
and move according to the chemoldne gradient. Therefore, (1) the simulating gradient 

1 5 must be correct and (2) the thymocyte must respond only to gradients it can currently 
interact with. 

To determine the right gradient, the scientific literature was surveyed to learn 
which chemoldne is expressed where and at what level. This mformation is available 
from different papers, ranging from papers whose subject is one specific chemokine 
20 and its expression in the thymus [32], to papers dealing with one specific area in the 
thymus and the expression of different chemokines in that area (e.g. [33]), to papers 
reviewing chemokine expression patterns in the thymus as a whole (e.g. [34]). The 
chemokine dataset was integrated to a four dimensional lattice where each dimension 
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Examples 
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Stands for flie concentration of one chemokine. Thymocytes first find out which of the 
gradients they should probe, calculate the relevant gradient and finally move. 

To find which of the gradients a thymocyte may now probe, the model 
preferably includes cell types as cell states. In the model (as in unmunology), cells are 
5 distinguished according to surfece maikers. The model considers which gradients are 
relevant at some specific stage. In other words, given a cell in a state characterized by 
the ejqpression of certain markers and given a certain gradient, where will the cell 
move? 

The scientific literature provides seven cell maikers as relevant for gradient 
10 decisions. Five of them may be either; "expressed" or '•un-ejdpressed". and two of them 
have an intermediate level of expression termed "low". The overall number of 
relevant states is therefore 2' x3» - 288- At run time, a cell scans through these 288 
states, finds the one it is in, and determines which chemokines it may respond to. 
During specification, each of these 288 states must be examined, to find an equivalent 
15 in scientific papers and to provide the biological meaning. During simulation, a 

decision tree is used to scan through the collection of possible states. Decisions (leafe 
of the last row) in the tree correspond to cell states. When the scan readies a 
conclusion (a leaf), the simulation generates events that tell the cell to which 
diemokine gradients it may now respond. 
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Example 2: Modeling epithelial cells 

Another example of specification is the inclusion of epithelial cells in the 
model. Epithelial cells in the thymus are stationary, yet their behavior is reactive and 
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changes continuously in response to various stimuli. The literature divides epithelial 
cells into many types. Since most of the woifc has been done using nucroscopy, the 
cell types are usually separated by their location and their size. To this microscopic 
division, tempoial behavior was added, which is the esqnression of different 
chemokines and cytokines in response to different events. For example, medullary 
epithelial cells have shorter processes (arms) than ottier epithelial cells and are usually 
no longer than 30 micrometer in lengdi. Medullary epithelial cells are considered the 
main elements in a process called negative selection, and have been therefore 
extensively measured for levels of expression of MHC class I and class n molecules. 

Epithelial cells were characterized as having not only a location but also a 
structure. The structure is the cell processes (arms). As thymocytes and other cells 
move through the thymus, they interact with the processes of epithelial cells. 

Specifying interaction 

When two cells meet during run time, directions are required to determine how 
their interaction should proceed. Researchers do not always know all the details of the 
interaction, and so they use different hypotheses to suggest possible outcomes of the 
interaction. The hypotheses and their suggested outcomes are referred to as **theories'' 
and outline them as objects with a behavior specified with statecharts. 

The statecharts of the instance are then executed and, according to different 
parameters, a conclusion of this interaction is reached. The conclusion may be the 
death of the T cell, instmctions to express one or another marker, instructions to 
express cytokines, instructions to proliferate, and more. Eventually, the instance 
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reaches the state maiked with "T", which means the instance is tenninated and will 
receive no further references. When another interaction of the same kind takes place, 
another instance of the same kind wUl be instantiated. Notice that many instances may 
co-exist, as the result of many Hiymocyte-epithelial ceU interactions occurring at the 
same time. According to a particular theory, a single epithelial cell may interact with 
many different T cells. 

Using Statecharts to communicate theories 

The diagrammatic nature of Statecharts makes them legible to scientists from 

different disciplines. To describe a theory with statecharts, a description given in text 

and non-formal diagrams is transformed into a rigorous, diagrammatic language. The 

resulting description is easy to communicate. 

By regarding theory as a separate component, it is possible to choose to plug in 

or unplug a theory on demand. A collection of available theories is built, and one is 
chosen. The choice of which tiieoiy should be instantiated may be made before the 
simulation is started. For example, all interactions between tiiymocytes and cortical 
epiflielial ceUs may be determined to follow one tiieory, while all other interactions 
follow a different theory. A choice of theory may also be made at run time, and the 
user can choose to switch between theories. The choice may also be made at run time 
by the simulation itself, when tiie right conditions develop. Theory, in the simulation, 
thus becomes interchangeable during the run. The simulation is only committed to the 
data, not to its interpretation. 
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The Front-End: An Interactive Ani mation 

WhUe the simulation runs, a firont-end to its activities is generated and 
presented to the user. The front-end is an interactive visual interface that embodies 
ceDs and molecules. The user can actually see what the cells and molecules are doing. 

The general setup 

TTie representation is a large collection of Flash™ movie clips that are the 
embodiment of agents and their states as they appear in the simulation running in 
Rhapsody. While the simulation is generating events and is changing the properties of 
tiie interacting agents, the simulation sends information about these changes to 
generate the Flash movie. The animation is generated on the fly. The animation is 
neitiier an end result of the simulation, processed at post-run, nor a preprogrammed 
movie. It is a Uving image capturing the look and feel of tiie physical image of the 
simulated cells and molecules during run-time. 

Figure 7 portrays in part (b) how one fliymocyte moves and in part (c) how an 
mteraction with an epitiielial cell takes place. Part (a) of the figure gives a snapshot of 
the running simulation. The figure shows collections of tiiymocytes around one 
q)ithelial cell m the animated user interfece. It is important to emphasize that <he 
image of die tiiymocytes is not a sketch made for tiie figure but a screen capture of the 
miming simulation. 

In parts (b) and (c) of the figure, a sketch of two mechanisms that determine the 
behavior of the cells is shown. In part (b), below tiie image of tiie tiiymocyte parts of 
the statechart of tiie thymocyte are shown. Only two sub-statecharts are shown, 
corresponding to tiie tiiree markers visible on the cell's surfece and not tiie fiiU 
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statechart, for the sake of clarity. The thymocyte currently expresses the receptors 
CD4 and CD8 (the immunological term is DP - Double Positive) and is responsive to 
the chemokine TECK. Contrary to the two markers for CD4 and CDS, which stand for 
real surface molecules wfli that name, the marker for TECK does not signify a 
5 molecule, but signifies the ability of the thymocyte to migrate according to a gradient 
created by fliat specific chemokine. This notation is used became the experimenters 
have only limited knowledge of which receptors cause which movements. The 
av£ulable data e:xperim^ters provide is of the form **which T cell migrates according 
to which chemokine" [14, 34, 37, 38, 39, 40], The sub-statecharts show how receptors 
10 are represented as orthogonal states. An expressed receptor will be in the state "hi^" 
and an unexpressed receptor will be in the state "low". On the left statechart, only one 
state is in "high". The state represents susceptibiUty to TECK migration. On the right 
side, two receptors are in "high" - CD4 dnd CDS. 

To be able to move, the thymocyte represented in the figure (as all other cells) 
15 continuously samples its environment. When the thymocyte finds a relevant 

chemokine gradient - a TECK gradient - it calculates the gradient difference across 
its sur&ce. Cell movement is directed accorduig to this calculation. In this example, 
the conclusion is for the thymocyte to move left 

Part (c) of Figure 7 portrays a diflferent mode of operation. The lower part of 
20 Figure 7 (a) ^ows a thymocyte next to part of the arm of an epithelial cell, 

represented as the two adjacent red diamonds. The thymocyte has just migrated from 
the right and touched the epithelial cell to its left. When the thymocyte and the 
epithelial cell meet, the conclusion of this specific interaction is the result of several 
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checks made during the execution of the statechart, 

thymocyte, the attributes of the thymocyte and the properties of the epithelial cell, and 
finally comes up with the conclusion tiiat, in this case, tiiie specific ttiymocyte should 
now proliferate. Proliferation will result m the creation of other thymocytes bearing 

5 the same markers and having flie same attributes as the parent cell. The proliferation 
updates the Flash movie. When a new thymocyte is created in the movie, an arrow to 
designate its ancestor appears and then vanishes. 

The simulation handles many such events during run-time. Hiymocytes 
continually move around in the simulated tiiymus, continuously check their 

10 envkonment for stimuli, respond to the stimuli, proliferate, mature, die, change their 
receptors, secrete cytokines and interact with other cells. All this is displayed at run- 
time on the user interface and in animated statecharts generated by Rhapsody. Since 
every agent in the simulation is in effect an instance in Rhapsody, tiie user may choose 
to focus on an animated statechart of the agent. Animated statecharts are useful when 

15 events and switches in states during simulation are to be studied. 

One may, for example, wish to follow the details of the interaction that resulted 
in migration towards the medulla. Since Rhapsody provides a "step-by-step" mode, 
we can interrupt the flow of the simulation at any time and continue one step at a time, 
while paying attention to relevant attributes and following any switches in states the 

20 cells go through. Choices made by 'theory" instances are followed, resulting in 
decisions. This course of action may be referred to as "debugging" the simulated 
biological process. This occurs at two levels. First, one may watch the visual 
embodiment of the simulation as it develops in the animated representation, to look 
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for emerguig pattexns, for dead-end paths, far undefined obswvables and for mistakes. 
To carefully scratinize parts and time bites, Ac power of animated statecharts is used 
to progress step-\sise. This allows every agent to be examined as one reacdve system 
and to handle tilie flood of incraning^ou^oing evoats in a controllable way. 

5 Interactivity 

Both flie visual user inter&ce and the underlying executed animated statecharts 
allow the user to manipulate flie simulation and to retrieve data. This is done in two 
separate ways. 

Interactions via user interface 

10 As we explained above, the front-end of the simulation is composed of a 

collection of movie clips. Each of the movie clips is in fact an interactive mraiu that 
allows the user to send data to the running simulation. Since the sent data is in fact an 
XML object (see "Materials and Methods'^, it is not limited in its contents. Available 
operations are perceived as belon^ng to one of two kmds: data manipulation or data 

IS request. 

Data mzmipulation 

Every object in the animation is also a clickable menu. Figure 8 dem6nsti:ates 
data manipulation and data request upon clicking the animated thymocyte. The menu 
item *1cill T cell" serves as an example of data manipulation. When the user clicks this 
20 item, the underlying executing simulation receives notification that it should now tell 
tiiis specific T cell to perform apoptosis (programmed deatii). The results of apoptosis 
are performed in the simulation itself. When the results are processed, the animation 
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wm recede the instmction from ihe simulation to now delete the thy^^ 
current view (and to perform any other representation tasks needed). 

Hie submenu I'Change Receptors" opens into four submenus that control the 
cell's recqitors (b). Part (c) of the figure shows the submenu that opens the menu item 
5 "Chemokine Receptors". By clicking any element in the checkbox table, the user can 
change the ability of the cell to migrate to any of the receptors. For example, upon 
clicking the checkbox in MDOYes. the ammation sends an event to the simulation. 
The^ulationwillthendotwothmgs: it will direct the cell that it may now migrate 
according to MDC and it informs the ammation that the thymocyte should now 
10 indicate that it is susceptible to MDC (by showing the MDC indicator). The user thus 
manipulates the simulation exactly in the way data manipulate the simulation. Data 
manqnilation events originating from the user are no different, as far as the simulation 
is concerned, from events that stem from data specification. 

15 Data retrieval 

In contrast to data manipulation, data retrieval events do not direct or drive the 
simulation process. Hie menu items "Link to Parent", ^Developmental Stage" and 
"Show TCR sequence" of part (a) in Figure 8 are examples of retrieval events. As 
seen on movie M3 (supporting online material), when the user clicks the menu item 
20 "Link to Parent" the animation retrieves a list of the cell's ancestors. With this list, the 
animation draws arrows indicating the path of the ancestors. The path will start with 
the selected thymocyte, draw an arrow to the thymocyte that gave it birth and 
iteratively follow the line of cells up to the primary thymocyte that started the lineage. 
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The present invention may optionaUy be used in many different types of 
implementations and appUcations, examples of which include, but are not 
limited to, analysis of any complex and dynamic data; teaching and course 
design; monitoring the evolution of any system; predicting system beha^dor in 
responseto intervention; for example. inbiological.ystems,healtii and disea^^ 

pharmaceutical development, system design, system modification, patient 
monUoring. etc; decentraUzation of presentation and analysis: flie animation 
may be viewed and studied at sites removed from tiie site at which the 
specifications are controlled; organizational planning; strat^c and tactical 
plamiinB in tiie military; war games, transport, communications, etc; transport 
and delivery system plamiing and analysis; any dynamic system fliat tiie human 
mind would understand better by animated >dsualization; any dyn^i^ jyg^^^^,- 
that needs specification, dtiier for design (to help build tiie system) or for 
analysis (to know how ttie system worics, e.g. biology) and would benefit fix>m 
a non-intrusive visualization. 

He present mvendon pnmdes «le« adv^rtages. «.d eleariy diltes over other 
attempted solutions ,h« are known in the art. For exan^le. for some computer games, 
more than one acUon may be perfonned at a given point in .he game. However, every 
single path is se, even though there may multiple paths. In odKr words, computer 
games only permit a finite number of predetermined paths. 
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In contrast, in the present invention, there are no preset paths. For example in 
a stateful system, the path taken by the reactive animation cannot be known in 
advance. From each state, a transition may be made to any one of a number of 
different states. With a relatively small number of states, the number of different 

5 possibilities becomes huge. When there are enough fectors wi&in a stateful system, 
there may be a combinatorial explosion and the number of paths may effectively 
become infinite, as the problem of analysis of the different paths becomes NP 
complete. The present invention overcomes this problem preferably by using a state 
engine, and decomposing the representation of the system to be ammated into a 

10 plurality of states, thereby avoiding the necessity to calculate predetermined paths. 
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WHAT IS CLAIMED IS: 

1. A method for producing animation of an object comprising: 
modeling a behavior of the object as a plurality of events; 
creating a visual d^iction at least of said plurality of events; 
detectmg an event associated with the object; and 

creating the animation according to s^d event with said visual depiction. 

2. A method according to claim 1, wherein said plurality of events 
comprises a plurality of temporal samples or a plurality of scenarios. 

3. A method according to claim 1, viierein said plurality of events 
comprises a plurality of states. 

4. The method according to claim 3, fiuther comprising: 
determining at least one transition between said plurality of states. 

5. The method of claim 4, wherein said at least one transition is determined 
according to at least one rule. 

6. The method of any of claims 3-5, wherein said creating said visual 
depiction further comprises creating a visual depiction of said at least one transition. 
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7. The method of any of claims 3-6, wherein said state represents an 
interaction between a plurality of objects. 



8. The method of any of claims 3-7, further comprising: 
interacting between a plurality of objects; and 

altermg a state of at least one object according to said interacting. 

9. The method of any of claims 3-8, fiirther comprising: 
receiving an ejctemal input; and 

altering a state of at least one object according to said external input. 

10. Tlie method of claim 9, wherdn said external input is provided through 
a user interface. 

11. The method of clahn 10, wherein said user mterface is for interacting 
witii a computer game. 

12. The method of any of claims 3-11, wherein said detecting said state is 
performed by a state engine, and wherein said creating the animation is performed by 
an animation engine, the method further compnsmg: 

receiving a command IBrom said state engine; 

parsing said command to determining said state of said object; and 
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translating said command to a format for said animation engine for areating the 
animation. 

13. A system for producing reactive animation of an object, wherein a 
behavior of the object is modeled as a plurality of events, comprismg: 

(a) an event drivoiengSne for modeling the plurality of states and at least 

one transition betweai the plurali^ of events; 

(b) an animation engine for creating a visual depiction at least of each of tiie 

plurality of events; and 

(c) an inteaface foe recdving an event associated with the object 
from the event driven engme, and for sending a command to said 
animation engine for creating said visual depiction according to said 
event 




14. The system according to claim 13, wherein: 
Said event driven engine comprises a temporal logic engine or a 
scenario based eng^e; and 

Said plurality of events comprises a plurality of time samples or a 
plurality of scenarios. 

15. The system according to claim 13, wherein: 
said event driven engine comprises a state engine; and 
said plurality of events comprises a plurality of states. 
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16. The system according to claim 15, further comprising: 
a plurality of statecharts; and 

a state processor. 

17. The system according to any of claims 13-16, wherein the 
animation engoie comprises: 

a plurality of animation pieces; 
a rendering engine; and 
an input translator. 

1 8- The system of claim 14, wherein said state engine con^rises 
Rhapsody™ and SMd animation engine comprises Flash™. 

19. Hie system according to any of claims 13-18, further comprising: 
(d) an external input module for sending a command to said interface for 

interacting with the object 

20. The system of claim 19, wherein said external input module 
comprises a user interface. 

21 . The system of claim 20, wherein the user interface operates in 
response to mouse clicks. 
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22. Hie system of claim 20. wherein the user interfece is comprised 
within the ammadon en^e. 

23. Hie system ofclaim 20 wherein the user mterfece is operatively 
associated with the animation engine. 

24. The system of claim 20 wherein the user interface is comprised 
within the interfece. 

25. The system of claim 20 wherein the user interface is operatively 
associated with tiie interface. 

26. A method for analyzing a biological system, the biological system 

featuring a plurality of biological components, the method comprising: 

providingdatarelatedtoaplurality of activities of the pluraUtyofbiolog^^^ 

components of the biologioal system; 

analyang fte <iata to fiwm at least one Sfpecification; 

c^cting a plurality of states and at least one transition for at least a portion 
of the pluraUty of biological components according to said at least one specification; 
and 

creating a visual depiction of said at least a portion of the plurality of biological 
components in each of said plurality of states. 
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27. The method of claim 26, ftnther comprising: 
detecting a state of at least one biological component; and 
creating animation according to said state with said visual depiction. 

28. The method of clahn 27, wherein the biological system comprises a 
thymus. 

29. A method for analyzing a biological system, the biological system 
featuring a plurality of biological components, the method conq>rising: 

providing datarelated to a pluraUty of activities of the plurality of biological 
components of the biolo^cal system; 

analyzing the data to form at least one specification; 

decomposing said at least one specification into a plurality of events for at least 
a portion of the pluraUty of biological components according to said at least one 

specification; and 

creating reactive animation of said at least a portion of the plurality of 
biological components, said reactive animation being at least partially detennined 
according to said plurality of events. 
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ABSTRACT 

A system and method for generating reactive animation involved in providing 
a generic link between tools for the specification and execution of reactive systems 
and tools for graphic animation. The preseat invention can optionally be used in a 
wide range of applications, including computer games, navigation and trafBc systems, 
physiology models, and interactive scientific animation. Reactive animation helps 
make the programnung of such applications more reliable, expetMous and natural. 
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Figure 5. Soicn capture and relevant sCatediart of thymocyte 
movement and piolireration 
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Figure 6. An example of menus diat open in response to dtcking a 
thymocyte. 



